Welcome

Welcome to EPsy 8264: Advanced Multiple Regression Analysis. This is an advanced seminar for doctoral students in education. The pre-requisites for this course are EPsy 8251 and EPsy 8252.


Instructor

Andrew Zieffler (zief0002@umn.edu)
Virtual Office: Zoom
Office Hours: Wednesday 9:00 AM–10:00 AM; and by appointment

Teaching Assistant

Vimal Rao (rao00013@umn.edu)
Virtual Office: Zoom
Office Hours: Tuesday 5:30 PM–6:30 PM; and by appointment


Classroom

  • Tuesday/Thursday (9:45–11:00): Zoom


Syllabus


Textbooks

The course textbooks are available via the University of Minnesota library (Fox) and online (Grolemund & Wickham).


Prerequisites

Prerequisite knowledge include topics from a basic statistics course:

  • Foundational topics in data analysis;
    • Design (e.g., random assignment and random sampling)
    • Descriptive statistics and plots
    • One- and two-sample tests

And, topics from EPsy 8251: Methods in Data Analysis for Educational Research I:

  • Statistical Computation
    • Using R
    • Data wrangling/manipulation
    • Plotting
  • Correlation;
  • Simple regression analysis;
    • Model-level and coefficient-level interpretation
    • Ordinary least squares estimation
    • Standardized regression
    • Partitioning sums of squares
    • Model-level and coefficient-level inference
    • Assumption checking/residual analysis
  • Multiple linear regression
    • Model-level and coefficient-level interpretation and inference
    • Assumption checking/residual analysis
    • Working with categorical predictors (including adjusting p-values for multiple tests)
    • Interaction effects

And topics from EPsy 8252: Methods in Data Analysis for Educational Research II:

  • Dealing with nonlinearity;
    • Quadratic effects
    • Log-transformations
  • Probability distributions;
    • Probability density
  • Maximum likelihood estimation;
  • Model selection;
    • Information criteria
  • Linear mixed-effects models (cross-sectional/longitudinal)
    • Basic ideas of mixed-effects models
    • Fitting models with random-intercepts and random-slopes
    • Assumptions
    • Likelihood ratio tests
  • Generalized linear models
    • Logistic models

For the topics listed, students would be expected to be able to carry out an appropriate data analysis and properly interpret the results. It is also assumed that everyone enrolled in the course has some familiarity with using R. If you need a refresher on any of these topics, see:


Calendar

Below is the tentative schedule for the class. The dates are subject to change at the instructor’s discretion.


Date Topic
  Sept. 08 Welcome to EPsy 8264
Unit 01: Mathematics
  Sept. 10 A Bit of Linear Algebra
  Sept. 15 Some Vector Geometry
  Sept. 17 Regression using Matrices
  Sept. 22
  Sept. 24 Matrix Decomposition
  Sept. 29
Unit 02: Regression Diagnostics
  Oct. 01 Regression Diagnostics
  Oct. 06
Unit 03: Dealing with Heteroskedasticity
  Oct. 08 Variance Stabilizing Transformations
  Oct. 13 Weighted Least Squares (WLS) and Sandwich Estimation
Unit 04: Dealing with Collinearity
  Oct. 15 Collinearity Diagnostics
  Oct. 20 Variable reduction
  Oct. 22
  Oct. 27 Biased Estimation and Shrinkage
  Oct. 29
Unit 05: Model Selection
  Nov. 03 NO CLASS— Election Day
  Nov. 05 Traditional Methods of Model Selection
  Nov. 12 Cross-Validation
  Nov. 17
Unit 06: Piecewise Modeling
  Nov. 24 Piecewise Regression
  Nov. 26 NO CLASS—Thanksgiving Break
  Dec. 01 Piecewise Regression
  Dec. 03 Spline Models
  Dec. 08
  Dec. 10
  Dec. 15 TBA

Assignments

Below are the due dates for the assignments, as well as links to the PDF file for each assignment. The due dates may change at the instructor’s discretion. Any revised due dates will be announced in class and posted to the website.


Assignment Due Date PDF
Assignment #1: Matrix Algebra for Linear Regression
[Updated 09-11-2020] 		Sept. 24
Assignment #2: Matrix Decomposition Oct. 06
Assignment #3: Regression Diagnostics Oct. 15
Assignment #4: Using WLS to Model Data with Outliers Oct. 22
Assignment #5: Variable Reduction
[Updated 10-14-2020] 		Nov. 10
Assignment #6: Shrinkage Nov. 24
Assignment #7: Cross-Validation Dec. 08
Assignment #8: Regression Splines Dec. 15

Data

Below are the links to the data sets and data codebooks used in the notes, scripts, and assignments.


Name Data Codebook
bluegills.csv
canadian-prestige.csv
contraception.csv
credit.csv
davis.csv
davis-corrected.csv
duff.csv
duncan.csv
education-expenditures.csv
equal-education-opportunity.csv
evaluations.csv
fake-piecewise-data.csv
galton.csv
grade_data.csv
houston.csv
iowa-judges.csv
loess.csv
mcycle.csv
mpls-violent-crime.csv
polynomial-example.csv
relate.csv
slid.csv
stack-1979.csv
tokyo-water-use.csv

Materials

Welcome to EPsy 8264

Prior to class:

After class:

  • If you will be using your computer for statistical computing (rather than the RStudio server), install (or update) R and RStudio onto your personal computer. See R and RStudio Installation and Setup for help.

A Bit of Linear Algebra (Sept. 10)

In this unit we will learn some of the basic linear/matrix algebra that is useful for regression analyses.

Prior to Class:

  • Read Fox (Sections 1.1–1.4)

In Class

Additional Resources

  • Burrill, G., Burrill, J., Landwehr, J. & Witmer, J. (1998). Advanced modeling and matrices Orangeburg, NY: Dale Seymour Publications. (This is a module from Data-Driven Mathematics written for secondary school students and teachers)
  • Irizarry, R., & Love, M. (2018). Biomedical data science PH525x series. (Chapter 4 addresses matrix algebra)
  • Namboodiri, N. K. (1984). Matrix algebra: An introduction. Thousand Oaks, CA: Sage.
  • Savov, I. (2017). No bullshit guide to linear algebra. Minireference Co.


Some Vector Geometry (Sept. 15)

In this unit we will look at some of the geometric ideas underlying vectors that are useful for regression analyses.

Prior to Class:

In Class

Additional Resources


Regression using Matrices (Sept. 17; 22)

In this unit we will learn some of the basic linear/matrix algebra that is useful for regression analyses.

Prior to Class:

  • Read Fox (Sections 4.1–4.6)

In Class

Additional Resources


Matrix Decomposition (Sept. 24; 29)

Prior to Class:

  • Read Fox (Sections 1.6)

In Class

Additional Resources


Regression Diagnostics (Oct. 01; 06)

Prior to Class:

In Class

Additional Resources

Variance Stabilizing Transformations (Oct. 08)

Prior to Class:

In Class

Additional Resources


Weighted Least Squares (WLS) and Sandwich Estimation (Oct. 13)

Prior to Class:

In Class

Additional Resources

  • Shin, H.-C. (1998). Weighted least squares estimation with sampling weights. Journal of Econometrics, 8(2), 251–271.
  • Solon, G., Haider, S. J., & Woolridge, J. (2013). What are we weighting for? (Working Paper No. 18859; NBER Working Paper Series). National Bureau of Economic Research.


Collinearity Diagnostics (Oct. 15)

Prior to Class:

  • Nothing

In Class

Additional Resources

  • N/A


Principal Components Analysis via Eigendecomposition (Oct. 20)

Prior to Class:

  • Nothing

In Class

Additional Resources

  • Many. Google “PCA” and “eigendecomposition”.


Principal Components Analysis via Singular Value Decomposition (Oct. 22)

Prior to Class:

In Class

Additional Resources